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1 . Introduction 

The  presence  of  random  noise  in  images  has  two 
main  undesired  effects  on  further  utilization  of  the 
images.  Firstly,  the  noise  makes  the  image  extremely  un- 
pleasant to  the  human  visual  system  (HVS) . Secondly,  it 
affects  the  performance  of  an  automatic  image  analysis 
system  by  giving  spurious  edge  responses  at  points  that 
really  contained  no  substantial  edges  in  the  original 
(noise-free)  image.  There  are  several  heuristic  as  well 
as  mathematically  optimal,  linear  as  well  as  non-linear 
noise  smoothing  methods  many  of  which  are  surveyed  in  two 
recent  books  on  picture  processing  [1  , 2 ] . Most  of  the 
linear  methods  have  space-invariant  smoothing  point  spread 
functions  (PSF)  and  as  a result  there  is  always  a trade-off 
between  the  degree  of  smoothing  of  noise  and  the  degree  of 
blurring  of  genuine  object  boundaries.  Unless  one  is 
willing  to  post-process  the  smoothed  picture  to  enhance  it 
and  sharpen  the  edges  [ 3 ] it  may  be  necessary  to  use  an 
edge  preserving  space-variant  smoothing  method. 

One  of  the  pioneering  concepts  in  space-variant 
smoothing  was  due  to  Graham  [ 4 ] . His  method  essentially 
involved  smoothing  a pixel  over  a neighborhood  that  belongs 
to  the  same  object  (or  background)  region  as  the  pixel  it- 
self. This  requires  determining  the  object  boundaries 
with  reasonable  accuracy.  Similar  methods  utilizing 
various  types  of  edge  detector  such  as  the  gradient,  the 
Laplacian  and  the  like  are  conceivable.  Examples  of  recent 
work  of  this  kind  are  decision-directed  noise  smoothing 
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by  Nahi  and  Habibi [ 5 1 and  iterative  enhancement  of  noisy 
images  by  Lev  et  al.  [ fi  ].  Another  concept  in  edge  pre- 
serving smoothing  is  the  median  filter  [ 7 ] . we  investi- 
gate here  the  feasibility  of  a different  approach  to  adap- 
tive smoothing.  The  investigation  is  carried  out  treating 
images  in  only  one  spatial  direction  but  the  extension  of 
the  method  to  two  dimensions  need  not  restrict  its  feasi- 
bility. The  proposed  smoothing  method  uses  a parametric 
smoothing  function,  to  be  described  in  detail  in  Section  4, 
the  parameter  being  constantly  updated  by  computing  a cer- 
tain local  measure  on  the  image.  The  definition  of  the 
local  measure  is  based  on  certain  stochastic  properties  of 
the  random  noise. 

The  second  effect  of  random  noise  in  images  can  now 
be  reduced  in  an  obvious  way,  i.e.,  by  detecting  edges  in 
the  adaptively  smoothed  image  rather  than  in  the  noisy 
image.  But,  as  it  turns  out,  the  local  measure  mentioned 
above  gives  us  another  way  of  obtaining  enhanced  edges. 

If  regions  that  have  no  edge  in  the  original  image,  but 
that  contain  random  noise  of  magnitude  great  enough  to 
give  false  edge  response,  can  be  identified,  then  con- 
ceivably the  edge  response  can  be  modified  in  such  re- 
gions to  yield  a more  desirable  edge  picture. 

Let  us  assume  that,  in  the  simplest  case,  the  original 
image  consists  of  regions  of  constant  gray  level,  with 
boundaries  between  pairs  of  regions  being  characterized 
by  abrupt  changes  in  the  gray  levels  ("step  edges").  An 
edge  operator  such  as  the  maximum  absolute  difference  of 
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A Simple  Model 


Let  us  consider  a scene  of  the  type  described  in  the 
previous  section  in  which  there  are  only  two  regions,  each 
of  constant  gray  level,  one  region  being  embedded  in  or 
surrounded  by  the  other.  The  gray  level  transition  from 
Sq  in  the  surrounding  region,  called  "background",  to  s^^ 
in  the  embedded  region,  called  "object",  is  assumed  to  be 
abrupt,  as  described  in  the  earlier  section.  Let  us  con- 
sider a single  row  of  this  scene  passing  through  the 
object.  The  gray  level  g(i)  as  a function  of  distance  i 
may  appear  as  shown  in  Figure  1.  In  the  presence  of  zero- 
mean  orthogonal  noise  n(i)  the  noisy  gray  levels  x(i)  in 
the  same  row  may  be  as  shown  in  Figure  2. 

Figures  3a-c  are  gray  level  profiles  as  viewed 
through  windows  of  size  k points  (k  assumed  odd) , centered 
so  that  the  window  is  completely  in  the  background  region, 
completely  in  the  object  region,  and  partially  in  the 
background  and  partially  in  the  object  region*,  respectively 
These  gray  levels,  x' (i) , are  given  by 

x' (i)  = x(i+i)  (la) 

x(i)  = g (i)  + n(i)  (lb) 

where  !L  is  the  location  of  the  center  point  of  the  window. 
Now  suppose  we  compute,  for  each  of  the  three  cases 


*While  Fig.  3c  and  the  corresponding  figures  in  the  sub- 
sequent analysis  show  background  to  object  transitions, 
the  existence  of  similar  object  to  background  transitions 
is  understood  and  implictly  assumed  in  the  analysis. 
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neighbors,  the  Roberts  cross  operator,  or  the  gradient 
[8,2]  will  yield  nonzero  response  only  at  the  region 
boundaries.  However,  the  introduction  of  additive  zero- 
mean  orthogonal  noise  to  the  noise-free  image  will  yield 
non-zero  responses  almost  everywhere. 

Now  suppose  that  we  have  a local  operator  which  can 
discriminate  in  the  noisy  picture  between  neighborhoods 
that  have  no  gray  level  transition  (no  "activity")  in  the 
original  image  and  neighborhoods  that  do  have  some  gray 
level  transition  (some  "activity")  in  the  original  image. 
This  "activity  detector"  will  enable  us  to  suppress  the 
edge  response  in  the  neighborhoods  of  no  activity  while 
retaining  or  enhancing  the  edge  response  in  the  active 
neighborhoods.  A local  image  operator  is  presented  here 
that  may  be  used  as  such  an  activity  detector. 
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Figure  1 

Object  and  background  gray  level 
transition  in  a noise-free  scene. 

a)  Background  to  object  transition 

b)  Object  to  background  transition 
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Figure  2 

Object  and  background  gray  levels  in  a 
noisy  scene. 
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Figure  3 

Gray  level  profiles  in  windows  of  size  k 
(j  = (k-l)/2) . 

a.  Window  located  completely  in  the  background. 

b.  Window  located  completely  in  the  object. 

c.  Window  located  partially  in  the  background 
and  partially  in  the  object. 


above, the  expected  gray  level  of  the  window  and  subtract 
it  from  the  gray  level  of  each  point  in  the  window.  The 
transformed  gray  levels  y(i)  are  given  by 

y(i)  = x'(i)  - E[x'(i)] 


= x(i+)i)  - E[x(i  + J-)] 


(2) 


The  transformed  gray  level  profiles  y(i)  in  the  windows 
corresponding  to  those  in  Figures  3a-c  are  shown  in 
Figures  4a-c,  respectively. 

In  Figure  4a  the  window  is  completely  located  in  the 
background,  therefore 

x(i+Jl)  = Sq  + n(i+£)  (3a) 

Hence , 

y(i)  = Sq  + n(i+Jl)  - E[sQ+n(i+£)] 

= Sq  + n (iti)  - Sq  ( n, 

= n(i+Jl) 


Therefore,  the  correlation  coefficient  P between  two 
neighboring  points  in  this  data  sequence  y(i)  is  given  b" 

p = E[y(i)y(i  + 1)  ]/E[y^(i)  ] 

= E [n  (i+i-)  n (i  + £.+l)  ] /E  [n^  (i+Jl)  ] 

, (3c) 

= 0/a^ 
n 

= 0 
2 

where  a is  the  variance  of  the  random  noise, 
n 

Similarly,  in  Figure  4b  the  window  is  completely 
located  in  the  object  region,  therefore 


= Sj^  + n (i+il) 


X (i+Z) 
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Figure  4 

The  transformed  gray  level  profiles  in  the 
windows  of  Figures  3a,  b,  and  c. 


and 


y(i)  = +n(i+!!.)  - E [ s ( i+2,)  ] 

= n(i+il)  (4b) 

which  again  yields 

p = 0 (4c) 

In  Figure  4c,  where  the  window  is  partially  in  the 
bac>cground  and  partially  in  the  object  region, 

x(i  + il)  = s(i  + Jl)  + n(i+£)  (5a) 

s(i+£)  = Sq+(s^-Sq)-U(^.')  (5b) 

where  U(>)  is  the  unit  step  function  and  Si'  is  a random 
point  (assumed  uniformly  distributed)  such  that 

£-j  < Z'  < Z+j.  (6) 

Let  the  expected  value  of  s(i+£)  be  p.  Then 

y(i)  = x(H-S,)  - E[x(i+S-)] 

= s(i+«,)  + n(i+il)-E[s(i+il)+n(i  + «.)  ] 

= s (i+H)  - ’p  -t  n (i+il) 

= s' (i+Z)  + n(i+Z)  (7) 

where  s' (i)  is  as  shown  in  Figure  5. 

Assuming  that  n(i+Ji)  is  uncorrelated  with  s'(i+;^h  the 
correlation  coefficient  in  the  window  of  Figure  4c  is 

= E[{s'  (i+£)+n(i+£)  Hs'  (i+«.+l) +n  (i+Jl+1)  } ] /E  [ { s ' ( i+C.) +n  ( i + «.)  (^] 
= E[s'  (i+i)s'  (i+«.+l)]/[a^  + E[s' ^ (i+Jl)  ] ] . (8a) 


It  can  be  shown  that  for  uniformly  distributed  Z'  the 
numerator  in  eqn.  (8a)  is  always  positive  and  hence 


For  uniform  £ ' and  large  k 


Pr{s(i+2.)  = s(i+i-+l)}  >>  Pr{s(i+jl)  f s{i  + )l+l)}  (8c) 

and  hence 

E[s'(i+Ji)  s'(i+S-+l)]  ~ E[s'^  (i+«.)  ] . (8d) 

When  the  noise  is  insignificant  (small  variance)  the 
above  p is  approximately  unity.  As  the  noise  variance 
increases  the  value  of  p goes  down,  approaching  zero  for 
very  large  noise  variance.  Hence  it  is  conceivable  that 
the  correlation  coefficient  of  sequences  of  windowed 
gray  levels  (with  the  mean  gray  level  of  the  window  sub- 
tracted from  the  points)  will  be  able  to  discriminate  be- 
tween neighborhoods  near  object  boundaries  and  neighbor- 
hoods away  from  object  boundaries  in  a noisy  environment. 

If  a window  contains  an  object  boundary  then  the  correlation 
coefficient  of  the  window  will  be  positive.  Thus  all  win- 
dows centered  within  ± j points  of  an  actual  object  bound- 
ary will  be  expected  to  have  high  correlation  coefficients. 
This  implies  that  a region  marked  by  a high  correlation 
coefficient  is  k points  wide,  even  though  the  actual 
boundary  is  a step  boundary  and  hence  is  only  one  point 
wide.  Hence  if  we  desire  to  keep  the  neighborhoods  marked 
by  high  correlation  coefficients  restricted  to  only  a few 
points  around  the  actual  object  boundary,  the  window  size  k 
must  be  reduced  accordingly. 


3 . Sample  Covariance  and  Complex  Scenes 

3 . 1 A measure  of  "mean  crossing" 

In  practice  the  correlation  coefficient  p has  to  be 
estimated  from  the  data  sequence  in  the  window.  A simple 
estimator  of  p is  obtained  via  the  sample  window  autocorre- 
lation (SWAC)  by 


This  estimate,  assuming  ergodicity,  approaches  the  actual 
value  of  p if  k is  large  [ 9 ] . 

For  y(i)  in  a neighborhood  away  from  the  boundary,  as 
in  Figure  4a  and  b,  y(i)  may  cross  zero  several  times  due 
to  the  presence  of  noise.  This  implies  that  often  y(i) 
and  y(i+l)  will  have  opposite  signs,  i.e.,  y(i)  is  nega- 
tive and  y(i+l)  is  positive,  or  vice  versa.  Thus.,  while 
the  denominator  is  always  positive,  the  numerator  in 
eqn.  (9)  will  contain  some  negative  terms  and  some  positive 
terms.  For  large  k the  positive  terms  may  nearly  cancel 
the  negative  terms  to  give 

p * p = 0.  (10) 

But  if  k is  small  then  p may  sometimes  have  value  less 
than  zero,  whereas  we  know  that  the  actual  correlation  co- 
efficient is  zero. 

On  the  other  hand,  in  a neighborhood  of  the  object 
boundary  there  may  be  fewer,  most  likely  only  one,  ze”'' 
crossings  and  hence  the  value  of  p here  will  be  greater 
than  zero.  Hence,  without  loss  of  any  desired  informati'^  •> 


the  value  of  p may  be  truncated  at  zero  to  make  the  estimate 
much  closer  to  the  actual  p when  k is  small.  In  such  a 
case  p will  always  yield  a value  between  0 and  1. 

Thus  in  general  if  the  sequence  of  gray  levels  x' (i) 
in  a window  (see  Figure  3)  crosses  the  mean  gray  level  of 
the  window  ("mean-crossing")  only  a few  times  relative  to 
k,  the  value  of  p will  be  high;  but  if  the  number  of  mean- 
crossings  is  higher,  the  value  of  p will  be  closer  to  zero. 
We  shall  denote  this  truncated  local  operator  p by  SWAC. 


3.2  Response  of  SWAC  to  a linear  boundar 


A linear  object  boundary  such  as  that  shown  in  Figure 
6,  where  the  transition  between  the  object  gray  level  and 
the  background  level  is  linear  with  distance,  is  a closer 
approximation  to  real-life  object  boundaries  than  a step 
edge.  In  the  absence  of  noise  the  response  of  the  sample 
auto-covariance  operator  to  a linear  boundary  can  be 
analyzed  as  follows.  Let  the  slope  (gray  level  change  per 
unit  distance)  of  the  boundary  be  2h  and  let  the  window  of 
size  k be  centered  at  the  center  of  the  boundary.  After 
subtracting  the  window  mean  the  sequence  of  gray  levels 
g(i)  will  have  values  as  shown  in  Figure  7. 

Let  the  width  of  the  boundary  be  defined  as 
w = (2d+l)  + 1 (11) 

= 2(d+l) 

where  d is  as  shown  in  Figure  7.  Equation  (11)  implies 
that  the  boundary  width  is  an  even  number  of  points  in  the 
case  being  considered.  V"Je  assume  that  w < k.  The  summation 


in  the  denominator  in  eqn.  (9)  is 

2 2 2 r 2 

I y^^(i)  = 2n(2(d+l)h)^  + 2 I (2hi)'^ 

i=-j  i=l 

= 8nh^(d+l)^  + 8h^  I 

i=l 


(12a) 


= k4h^(d+l)^  - (2d+l) 4h^ (d+1) ^ + 8h^  ll^tilildj^) 

o 

2 

= kh^w^  - h^w^(w-l)  + ^ (w-2) (w-l)w 


= kh^w^  - j h^(w-l) (w+l)w 


(12b) 


where  n is  as  shown  in  Figure  7.  The  summation  in  the 
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Figure  6 


Linear  gray  level  transition  at  the  object 
boundary . 


Figure  7 

Gray  levels  at  k points  synunetrical ly  around 
the  center  of  a boundary,  k = 2n+2d+l. 


numerator  in  eqn.  (9)  is 


j-1  , ^ ? 

I y(i)y(i+l)  = 2 (n-1) (2 (d+1) h) ^ + 2 I 4h  i(i+i) 
i=-j  i=l 


= 8nh^ (d+1) ^-8h^ (d+1) ^ + 8h^  I + 8h^  I i. 

i=l  i=l 

Using  eqn.  (12a)  the  above  reduces  to 
!(•)  = I y^(i)  - 4h^(d+l) (d+2) 

i=-j 


= I y^(i)  - h^w(w+2)  (13b) 

i=-j 


Combining 


eqns.  (12)  and  (13)  we  get 

j~l  ^9 

I y(i)y(i+l) ]/[  I y^(i)] 

i=-j  i=-j 


k 

k-1 


1- 


3h^w  (w  t-2) 

31ch^w^  - 2h^w(w.-l)  (w+1) 


k 

k-1 


3(w+2)  , 

3)cw  - 2(w^-l) 


(14) 


It  turns  out  that  we  get  the  same  expression  for  p when  w 
has  an  odd  number  of  points.  It  is  obvious  from  eqn.  (14) 
that  the  response  p to  a linear  object  boundary  is  not  a 
function  of  the  slope  of  the  boundary.  For  this  reason 
3 will  perform  poorly  as  an  edge  detector  per  se.  Also, 
if  the  bacJcground  (or  object)  region  in  the  original  image 
contains  slowly  but  linearly  varying  gray  level  instead 
of  constant  gray  level  then  the  response  of  p in  the  back- 
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(13a) 
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ground  (or  object)  region  will  be  as  high  as  at  the  linear 
object  boundary.  In  the  presence  of  noise,  however,  if  the 
change  in  background  (or  object)  region  gray  level  is  small 
enough  compared  to  the  amplitude  of  the  noise  oscillations, 
then  there  will  be  enough  mean-crossing  to  lower  the  re- 
sponse p.  It  is  assumed  that  the  slope  of  the  object 
boundary  is  high  enough  so  that  the  amplitude  of  noise 
oscillation  does  not  introduce  too  many  additional  mean- 
crossings  to  lower  the  response  there.  Thus,  it  is  ex- 
pected that  even  v/hen  the  object  boundary  is  a rarnp,  and 
the  background  and  the  object  regions  are  slowly  varying 
rather  than  constant,  the  sample  auto-covariance  operator 
will  be  able  to  discriminate  noisy  oscillations  in  regions 
away  from  the  object  boundary. 

Table  1 shows  the  variation  of  p with  w for  various 
values  of  k.  As  it  appears  from  the  table  for  a fixed  k 
the  value  of  p does  not  change  appreciably  for  different 
boundary  widths.  It  is  also  evident  from  the  table  that 

t 

for  a given  boundary  width,  p increases  with  k.  Hence, 
to  obtain  a high  response  at  a boundary,  a large  k should 
be  chosen.  On  the  other  hand,  to  keep  the  effect  of  a 
boundary  on  the  SWAC  response  restricted  to  a small  neigh- 
borhood around  the  boundary  k should  be  kept  small.  It 
turns  out  that  k=3  always  results  in  3 = 0 except  when 
the  three  points  in  the  window  have  the  same  gray  level. 
This  degenerate  case  will  be  avoided  in  experimentation. 


4 . Adaptive  Noise  Smoothing 


This  section  describes 
The  smoothed  image  s(i)  is  g 

j-1 

s(i)  = x(i-£)h.(J.) 

i=-3  ^ 


the  proposed  smoothing  method, 
iven  by 

(15) 


where  x(*)  is  the  gray  level  of  the  noisy  image  and  h^(il) 
is  a weighting  function  that  depends  on  (i).  For  a given 
point  (i)  we  assume  that  the  v/eighting  function  is 


h(Jl) 


A e 


-h\Z\ 


(16) 


where  A is  a normalizing  constant  that  assures  that  the 
function  h(*)  is  unimodular,  and  b is  an  as  yet  undeter- 
mined parameter  whose  value  depends  on  (i).  For  a given  b 
the  smoothing  function  is  purposely  designed  to  resemble 
the  impulse  response  h(t)  of  a low  pass  R-C  filter  given 
by 

h(t)  = be”*^^U(t)  (17) 

where  b is  the  reciprocal  of  the  R-C  circuit  time  constant 
and  U(‘)  is  the  unit  step  function,  required  because  of  the 
causality  restriction.  Since  image  processing  does  not 
have  any  causality  restriction,  h(S-)  in  eqn.  (16)  has  been 
made  a doubly  exponential  function  (for  symmetry)  while  h(t) 
in  eqn.  (17)  is  one-sided  exponential. 

The  parameter  b in  eqn.  (16)  is  given  by 

b = -ln(l-p)  (18) 

where  p is  the  response  of  the  SWAC  operator  at  the  point 
(i) . In  using  the  definitions  of  eqn.  (16)  and  (18)  it  is 


assumed  that,  when  p=l  and  1=0 


(1-p) 


Lxm  ,1  A.  I 0 


(19) 


= 1 

even  though  0^  is  actually  indeterminate.  Thus,  if  we  de- 
note by  p(i)  the  response  of  the  SWAC  operator  at  the 
point  (i) , then 

h^(^)  = A{l-p(i)  } I (20a) 

and 

i i£i 

A = 1/  I a-p(i)  } ' (20b) 

£=-j 

Figure  8 shows  the  function  h^(£)  without  the  normalizing 
constant  for  various  values  of  p.  As  can  be  seen  from  the 
figure,  when  3(i)  is  small,  implying  a noisy  background  (or 
object)  point,  the  function  is  relatively  flat.  In  the 
limiting  case  when  p is  zero  h^(£)  is  constant  at  all  points, 
indicating  that  the  smoothed  gray  level  is  the  local  un- 
weighted average  of  all  the  k points  in  the  window  ("hard 
smoothing").  On  the  other  hand,  when  p(i)  is  high,  implying 
that  the  point  (i)  is  in  a relatively  clean  region  or  in  a 
region  with  few  mean-crossings,  the  function  h^(£)  is  rela- 
tively narrow.  In  the  limiting  case  when  p(i)  is  unity, 
h^(£)  is  unity  at  the  origin  and  zero  elsewhere,  indicating 
that  the  smoothed  gray  level  is  the  same  as  the  original 
gray  level.  It  may  be  noted  here  that  p(i)  is  unity  only 
when  all  the  points  in  the  window  have  the  same  gray  level 
and  not  when  the  window  is  on  some  object  boundary. 


5 . Experimental  Results 


Four  pictures  having  grayscale  0 to  63  and  .'ize  64x64 
were  selected  as  test  images.  The  first  two  pictures  show 
a square  object  and  some  chromosomes.  The  last  two  pic- 
tures are  forward  looking  infrared  (FLIP)  images  of  a tank 
and  a truck.,  respectively.  The  four  test  pictures  are 
shown  in  the  top  row  of  Figure  9.  The  picture  "square"  is 
a computer  simulated  square  object  of  constant  gray  level 
embedded  in  a constant  gray  level  background,  with  a linear 
boundary  between  the  object  and  the  background,  and  zero- 
mean  pseudorandom  noise  added.  The  four  pictures  have 
various  degrees  of  noise,  boundary  sharpness,  and  boundary 
separation  (distance  between  the  center  points  of  two  near- 
est boundaries  in  the  same  row. 

Next,  an  edge  operator  was  applied  to  these  four  pic- 
tures. The  edge  operator  chosen  is  the  DIFF  operator,  with 
2x2  neighborhood,  of  Hayes  and  Rosenfeld  [10].  It  takes 
the  absolute  difference  between  the  averages  of  two 
horizontally  touching  neighborhoods  of  size  2x2  as  the  edge 
value  e(i,j).  Note  that  the  DIFF  operator,  unlike  the 
SWAC  operator,  assumes  the  image  to  be  two-dimensional. 
There  are  many  edge  operators  that  smooth  out  the  input 
picture  before  computing  edge  values  in  order  to  reduce  the 
effect  of  noise  in  the  edge  picture.  Such  an  operator  was 
purposely  not  chosen  here  since  it  was  desired  to  investi- 
gate the  performance  of  the  SWAC  operator  in  an  environment 
in  which  the  edge  response  is  noisy. 

The  edge  responses  for  the  four  test  pictures  are 


shown  in  the  second  row  of  Figure  9.  As  is  obvious  from 
Figure  9,  the  edge  operator  is  certainly  one  that  is  sensi- 
tive to  noise  in  the  input  image.  The  bottom  row  of  Figure 
9 shows  the  same  four  edge  outputs  after  suppressing  the 
nonmaximn  over  a four-point  horizontal  neighborhood.  The 
nonmaximum  suppression  is  done  to  thin  the  edges.  All  the 
edge  pictures  in  Figure  9 and  subsequent  figures  are 
stretched  to  the  full  grayscale  range  of  0-63.  In  Figure 
9 as  well  as  in  subsequent  figures  the  horizontal  sizes  of 
the  various  pictures  seem  to  be  smaller  than  64  and  to  vary 
from  picture  to  picture.  The  reason  for  this  is  that  the 
response  of  a local  neighborhood  operator  is  not  well  de- 
fined near  the  border  of  a picture  where  the  neighborhood 
falls  partially  outside  the  picture.  In  the  case  of  some 
operators  (e.g.,  SWAC) , the  responses  at  these  border 
points  are  assumed  to  have  the  default  value  (=0) ; in  the 
case  of  other  operators  (e.g.,  DIFF) , these  border  points 
are  discarded  and  the  picture  size  is  reduced.  Thus  in 
the  second  and  third  rows  of  Figure  9 only  60  columns  are 
displayed;  the  four  border  columns  have  ill-defined  re- 
sponses. For  the  sake  of  convenience  in  display  the  test 
pictures  in  the  top  row  of  Figure  9 are  also  trimmed  to  60 
columns  even  though  all  64  columns  were  used  in  all  pro- 
cessing (e.g.,  DIFF  and  SWAC). 

The  SWAC  responses  for  the  test  pictures  were  then 
computed  using  various  window  sizes  k.  The  window  sizes 
compared  were  5,  7,  9,  11;  odd  numbers  were  chosen  as  win- 
dow sizes  to  keep  the  digital  center  of  the  window  unique. 


Figure  9 


The  four  test  pictures  and  the  edge 
responses . 

row  1:  The  test  pictures, 

row  2:  The  edge  (2x2  DIFF)  response  for 

the  test  pictures. 

row  3:  Local  maxima  over  4 horizontal 

neighbors  of  the  DIFF  pictures, 
column  1:  the  square 

column  2:  the  chromosomes 

column  3:  the  tank 

column  4 : the  truck 
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The  SWAC  image  for  each  of  the  above  four  values  of  k are 
shown  in  Figure  10.  For  k=ll  the  object  boundaries  are  out- 
lined in  the  SWAC  pictures  of  the  square,  the  chromosomes 
and  the  truck  by  very  thick  curves  (first  row  of  Figure  10) . 
It  may  be  mentioned  here  that  since  the  SWAC  operator  is 
one-dimensional  and  operates  on  a row  of  picture  at  a time 
the  horizontal  components  of  the  boundaries  are  not  out- 
lined in  the  SWAC  images.  The  reason  for  getting  thick 
outlines  of  the  boundaries  is,  as  mentioned  in  Section  2, 
that  the  response  due  to  a boundary  is  expected  to  spread 
j points  to  the  left  and  to  the  right  of  the  actual  bound- 
ary. For  this  reason,  in  the  SWAC  responses  of  the  tank 
where  boundary  separations  are  smaller,  the  responses  from 
various  boundaries  overlap  and  hence  do  not  outline  the 
actual  object  boundaries. 

The  noise  smoothing  of  the  test  pictures  was  done  by 
using  the  SWAC  response,  p,  displayed  in  Figure  10  and  the 
smoothing  function  of  eqn.  (20),  Obviously,  for  each  k, 
eqn.  (20)  yields  one  set  of  smoothing  functions  and  this 
yields  one  set  of  smoothed  images.  Figure  11  shows  the 
smoothed  test  pictures  for  k = 5,  7,  9,  11.  It  is  clear 
from  Figure  11  that,  for  all  the  four  values  of  k,  the  test 
pictures  are  adequately  smoothed  without  any  visible  de- 
gradation of  the  object  boundaries.  The  only  visibly  un- 
pleasant effects  in  the  smoothed  pictures  are  the  occas- 
sional horizontal  "streaks"  which  are  due  to  the  fact  that 
the  smoothing  operation  is  one-dimensional.  To  highlight 
the  performance  of  the  adaptive  smoothing  method  the  adap- 
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Figure  10 

The  SWAC  responses  for  various  k’s  and  various 
test  pictures. 


row  1 

k=ll 

row  2 

k=9 

row  3 

k=7 

row  4 

k==5 

column  1 

the  square 

column  2 

the  chromosomes 

column  3 

the  tank 

column  4 

the  truck 
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Figure  11 

Adaptive  smoothing  of  the  test  pictures  for  various  k's. 

row  1:  k=ll 

row  2:  k=9 

row  3:  k=7 

row  4:  k=5 

column  1:  the  square 

column  2:  the  chromosomes 

column  3:  the  tank 

column  4 : the  truck 


tively  smoothed  pictures  for  k=ll  are  compared  with  the 

test  pictures  smoothed  by  a local  unweighted  averaging,  a 

common  method, over  the  same  neighborhood  size.  The  results 

for  adaptive  smoothing  and  local  averaging  are  shown  one 

above  another  in  the  top  two  rows  of  Figure  12.  The  bottom 

two  rows  of  Figure  12  show  the  result  of  edge  detection, 

using  the  same  dIFF  operator  as  in  the  middle  row  of  Figure 

9.  The  relatively  thin  and  noise-free  edges  in  the  third  ! 

row  of  Figure  12  are  due  to  adaptive  smoothing.  The  thick 

edges  in  the  bottom  row  of  the  same  figure  are  due  to  local 

averaging.  Also  evident  from  th^e  bottom  two  rows  of 

Figure  12  is  the  fact  that  the  highest  edge  responses  in 

the  averaged  pictures  are  not  as  strong  compared  to  the 

false  responses  in  the  backgrounds  as  they  are  in  the  case  i 

I 

of  the  adaptive  smoothing.  | 

I 

i 

A more  detailed  comparison  of  the  adaptive  smoothing 
method  with  local  averaging  or  median  filtering  is  de- 
sirable. Where  an  automatic  image  analysis  system  is  used 
it  may  be  of  interest  to  compare  the  improvement  in  the 
performance  of  the  system  under  the  various  smoothing  tech- 
niques. A theoretical  analysis  of  the  performance  of  the 
adaptive  smoothing  method  v/ould  also  be  desirable.  The 
criterion  of  performance  must  take  into  account  error  in 
the  edge  valaue  as  well  as  the  error  in  the  gray  level  of 
the  output  picture. 


Figure  12 

Comparison  of  the  adaptive  smoothing  and  local  averag- 
ing for  k=ll. 

row  1:  The  test  pictures  smoothed  adaptively, 

row  2:  The  test  pictures  locally  averaged, 
row  3:  The  edge  (DIFF)  responses  for  the  pic- 
tures in  row  1. 

row  4:  The  edge  (DIFF)  responses  for  the  pic- 

tures in  row  2. 

column  1 : the  square 

column  2 ; the  chromosomes 
column  3:  the  tank 

column  4 : the  truck 
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Figure  13 

Nonmaxima  suppressed  SWAC  responses  for  various  k*s. 

row  1:  k=lL 

row  2:  k=9 

row  3:  k=7 

row  4:  k=5 

column  1:  the  square 

column  2:  the  chromosomes 

column  3:  the  tank 

column  4 : the  truck 


To  investigate  the  possiblity  of  SWAC  as  an  edge 
operator,  local  nonmaximum  suppression  was  performed  on 
the  SWAC  images.  The  result  is  shown  in  Figure  13.  When 
an  edge  thinning  operation  such  as  nonmaximum  suppression 
is  applied  to  the  SWAC  pictures  it  yields  many  spurious  re- 
sponses in  the  background,  and  in  all  except  the  chromo- 
some pictures  it  produces  sets  of  discontinuous  edge 
points,  but  no  sharp  thin  outlines  at  the  object  bound- 
aries. In  the  case  of  the  chromosomes,  where  the  boundaries 
are  sharp  and  the  background  is  noise-free,  non-maximum 
suppression  yields  thin  connected  curves  outlining  the 
object  boundaries.  The  same  phenomenon  may  also  be 
observed  for  k = 5,  7,  and  9. 

Comparing  various  rows  of  Figure  10  it  may  be  observed 
that  while  a smaller  k gives  thinner  object  boundary  out- 
lines. it  also  gives, as  predicted  in  Section  3.2,  weaker  re- 
sponses at  the  object  boundaries  (see,  for  example,  the 
chromosomes  picture  ) and  large  streaks  of  spurious  high 
response  (the  white  lines  in  the  Figures)  in  the  back- 
grounds. The  spurious  streaks  are  due  to  the  fact  that 
often  there  are  small  sets  of  neighboring  points  in  the 
backgrounds  that  have  a constant  gray  level  and  hence 
appear  noise-free  in  a smaller  window,  but  increasing  the 
window  size  brings  in  more  neighboring  points  that  make 
the  noise  evident. 

A weighted  edge  response  for  each  of  the  test  pictures 
was  computed  next  by  using  the  SWAC  response  at  a point 
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Figure  14 

The  weighted  edge  responses  for  various  test  pictures 
and  various  k 's. 


row  1 
row  2 
row  3 
row  4 


k=ll 

k=9 

k=7 

k=5 


column  1 
column  2 
column  3 
column  4 


the  square 
the  chromosomes 
the  tank 
the  truck 
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(i,j)  as  the  a posteriori  weighting  coefficient  of  the  edge 
response  e(i,j)  at  that  point.  The  effect  of  this  weight- 
ing is  expected  to  be,  as  mentioned  earlier,  to  attenuate 
the  noisy  responses  in  the  noisy  background  regions  and  to 
relatively  enhance  the  edge  responses  in  the  neighborhoods 

of  the  object  boundaries.  The  results  of  this  weighting  , 

for  various  values  of  k are  shown  in  Figure  14.  The 
weighted  edge  gives  fairly  high  responses  at  the  object 

boundaries  and  relatively  insignificant  false  responses  in  ! 

the  background.  In  the  chromosomes  picture,  where  the  un- 
weighted edge  picture  is  practically  noise-free,  the 
weighted  edge  pictures  also  have  sharp  noise-free  edges  at 
the  boundaries.  Thus,  while  the  SWAC  operation  discriminates 
against  noise,  it  has  no  adverse  effect  on  noise-free  images. 

There  is  no  trade-off  in  SWAC  performance  as  in,  for  ex-  1 

ample,  edge  detection  by  the  absolute  difference  of  window 

means,  where  the  effect  of  background  noise  is  reduced  at 

the  cost  of  thickening  the  edge  responses  at  the  object 

boundaries. 

The  SWAC  response  may  also  be  used  as  an  a posteriori 
weighting  coefficient  of  nonmaxima  suppressed  edge  (DIFF) 
responses.  Figure  15a  shows  the  result  of  doing  this  for 
k = 5,  7,  9,  and  11.  Even  though  many  of  the  edges  in 
Figure  15a  are  thin,  and  of  course  noise-free,  there  seems 

to  be  a high  risk  of  loosing  continuity  in  the  edges.  i 

Figures  15b  and  15c  show  the  result  of  using  nonmaxima  ' 

suppressed  SWAC  responses  as  the  a posteriori  weighting 
coefficients  of  edge  pictures  and  nonmaxima  suppressed  edge 
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(a) 


Figure  15 

i 

I 

The  effects  on  weighted  edge  response  of  nonmaxima  j 

suppression  at  various  stages.  * j 

(a)  nonmaxima  suppression  of  the  unweighted  edges 

(b)  nonmaxima  suppression  of  the  SWAC  responses 

(c)  nonmaxima  suppression  of  both  the  unweighted 
edges  and  the  SWAC 
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pictures,  respectively.  Among  Figures  14,  15a,  15b,  and 
15c,  Figure  14  seems  to  have  the  most  desirable  edge  pic- 
tures , 

It  is  conceivable  that  the  false  edge  responses  in 
noisy  backgrounds  (or  object  interiors)  can  be  ;:u[  pressed 
by  some  method  other  than  a measure  ot  the  degree  of 
"noisyness"  in  a neighborhood.  A segmentation  procedure 
based  on  statistical  hypothesis  testing  [11]  was  used  to 
determine  if  a neighborhood  of  a certain  size  around  each 
point  belongs  to  a single  region,  such  as  background,  or  to 
more  than  one  region,  as  when  the  point  is  at  or  near  an 
object  boundary.  The  four  test  pictures  segmented  by  the 
above  hypothesis  test  are  shown  in  Figure  16.  In  Figure 
16a  the  dark  points  (gray  level  zero)  are  where  the  null 

hypothesis,  that  a neighborhood  of  size  kx  k around  the 
point  belongs  to  a single  population,  was  rejected.  The 
confidence  level  and  the  ratio  of  the  number  of  intervals 
to  the  sample  standard  deviation  of  the  neighborhood  were 
0.99  and  1.8,  respectively  (see  [11]  for  the  meanings  and 
the  importance  of  these  quantities) . Figure  16b  is  the 
negative  of  Figure  16a.  The  top,  the  middle,  and  the 
bottom  rows  of  Figure  16  correspond  to  neighborhood  sizes 
of  11x11,  9x9,  and  7x7,  respectively.  When  the  gray  levels 
in  Figure  16b  are  used  as  the  point-wise  weighting  co- 
efficients to  construct  weighted  edge  pictures  from  DIFF 
responses  (Figure  9,  2nd  row)  it  results  in  very  insignifi- 
cant, if  any,  noise  suppression  and  edge  enhancement.  The 
resulting  weighted  edge  pictures  are  shown  in  Figure  17. 
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Segmentation  by  the  hypothesis  test. 

(a)  dark  points  are  rejected  points 

(b)  the  negative  of  (a) 

row  1:  neighborhood  size  11x11  column  1 

row  2;  neighborhood  size  9x9  column  2 

row  3:  neighborhood  size  7x7  column  3 

Column  4 


the  square 
the  chromosomes 
the  tank 
the  truck 
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Figure  17 

The  weighted  edge  pictures  using  the  hypothesis 
test  with  kxk  neighborhoods. 

row  1:  k=ll 

row  2 : k=9 

row  3:  k=7 

column  1:  the  square 

column  2:  the  chromosomes 

column  3:  the  tank 

column  4:  the  truck 


6. 


Conclusions 


The  performance  of  the  adaptive  smoothing  method  on  the 
test  pictures  is  excellent  in  the  sense  that  the  smoothed 
pictures  have  noise  suppressed  extremely  well  and  have  no 
apparent  degradation  of  object  boundaries.  The  thick- 
nesses of  the  edges  detected  by  the  DIFF  operator  before 
and  after  apative  smoothing  are  seen  to  be  the  same.  The 
smoothing  method  is  based  on  a very  simple  concept  of 
measuring  the  degree  of  noisiness  in  a neighborhood.  The 
SWAC  noise  measure  is  motivated  by  the  stochastic  property 
of  the  noise,  namely  that  it  is  orthogonal.  Even  though 
the  sample  size,  k,  is  too  small  from  the  point  of  view  of 
statistics,  the  effectiveness  of  the  measure  in  guiding  the 
adaptive  smoothing  function  is  impressive.  The  degree  of 
smoothing  is  reduced  not  only  near  the  object  boundaries, 
as  in  Graham's  method  [ 4 ],  but  also  in  regions  that  do  not 
contain  appreciable  noise.  Even  if  a very  simple  image 
model  is  used  in  the  theoretical  analysis,  the  effectiveness 
of  the  smoothing  method  seems  to  extend  to  almost  any  pic- 
ture. The  method  does  not  require  any  a priori  knowledge, 
such  as  noise  variance,  or  any  knowledge  of  statistics  of 
edge  values  in  the  picture  that  may  be  required  by  an  edge- 
detection  oriented  smoothing  method  (e.g.,  [4  ])•  It 

appears  that  variation  of  k,  the  operator  size,  does  not 
make  much  difference  in  the  smoothed  pictures. 

The  operator  SWAC  also  gives  us  a systematic  way  of 
reducing  false  response  in  edge  pictures.  The  response  of 
the  SWAC  operator,  when  used  as  an  a posteriori  weighting 


of  an  otherwise  noisy  edge  detector,  discriminates  against 
background  noise  in  noisy  pictures  and  enhances  genuine 
edge  responses  at  the  actual  object  boundaries  without 
thickening  the  boundary  outlines  or  having  any  adverse 
effect  on  noise-free  images.  It  appears  that  there  is  not 
much  difference  in  the  weighted  edge  pictures  for  various 
window  sizes.  A smaller  window  size,  while  involves  less 
computation,  outlines  the  well-separated  object  boundaries 
with  relatively  thinner  lines.  It  should  be  borne  in  mind 
that  the  types  of  scenes  considered  in  the  analysis  as  well 
as  in  the  experiments  do  not  have  any  textural 
detail . 

It  may  be  argued  that  the  false  response  of  any  edge 
detector  can  be  reduced  by  weighting  it  with  the  response 
of  another  "orthogonal"  edge  detector.  Such  a method  will 
solely  rely  on  the  fact  that  the  false  responses  of  two 
different  edge  operators  may  (hopefully)  not  lie  at  the 
same  points.  On  the  other  hand  a method  based  on  gray 
level  statistics  will  require,  and  be  valid  over,  large 
neighborhoods.  One  disadvantage  of  large  neighborhood 
size  is  the  loss  of  resolution,  such  as  in  the  top  two  rows 
of  Figures  16a  and  16b,  exemplifying  the  result  of  the 
hypothesis  test.  This  is  one  disadvantage  that  the  method 
based  on  SWAC  rio's  not  suffer  from. 
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